agenda for course:

• week 1 – R workflow, navigation, programming basics
• week 2 – working with datasets and external files, data cleaning + manipulation
• week 3 – summarizing data with dplyr::, visualizing data with ggplot2::
• week 4 – document authoring with R Markdown, working with the web

course materials will eventually all be on the course website:

http://lefft.xyz/r_minicourse

each week we'll have slides, notes, and a script. little exercises will be interleaved throughout the notes. the best way to write up solutions is to start a new R script called (e.g.) week1_exercises.r and type directly into that.

there will also be a list of links to useful resources up on the site

R scripts

• a plain-text file with extension .R or .r
• all plain-text files (e.g. .txt) can be opened and edited directly in any text editor
• contains R code that we'll run interactively in R Studio
• also contains comments, which are just annotations that explain what the code is doing

datasets

• all kinds of extensions, e.g. .csv, .tsv, .xls, .xlsx, .dat, .sav, .dta. nowadays, R can read them all. we'll go through examples of several in week 2.
• working with .csv files is generally preferable, since they are simple and come in plain-text format.
• proprietary formats like .xlsx have certain nice features, but they're binary files, which can make their behavior unpredictable (and depend on the Excel version used to create them).
  
• a less common format is .Rdata/.rda, which contains an R workspace with datasets and objects pre-loaded. (not plain-text so I try to avoid them)

R Markdown files

• extension .Rmd or .rmd
• plain-text format (opens in any text editor)
• a special kind of R script from which nice, clean documents can be easily generated (in .pdf, .html, or .docx formats)
• easiest way to compile is with cmd+shift+k from R Studio

when you're using R, it's "looking" in a specific directory (folder). many tears have been shed over trying to get R to look in the desired directory (mine and those of countless other victims).

the best way to start an R session is to grab/make a plain text file with extension .r (e.g. my_script.r), put it in its own folder (e.g. R_folder), and then open it with R Studio (which you should set as the default).

if you start R by opening a specific script in R Studio, R will be looking into the folder containing your script and you won't have to mess with working directories.

you can also to go "tools" –> "global options" –> "default working directory" within R Studio to tell R where it should look if you just open R Studio directly

navigating R Studio

time to start writing code!

# welcome to the R mini-course. in keeping with tradition...
print("...an obligatory 'hello, world!'")

## [1] "...an obligatory 'hello, world!'"

# this line is a comment, so R will always ignore it.
# this is a comment too, since it also starts with "#".

# but the next one is a line of real R code, which does some arithmetic:
5 * 3

## [1] 15

# we can do all kinds of familiar math operations:
5 * 3 + 1

## [1] 16

# 'member "PEMDAS"?? applies here too -- compare the last line to this one:
5 * (3 + 1)

## [1] 20

# usually when we do some math, we want to save the result for future use.
# we can do this by **assigning** a computation to a **variable**
firstvar <- 5 * (3 + 1)

# now 'firstvar' is an **object**. we can see its value by printing it.
# sending `firstvar` to the interpreter is equivalent to `print(firstvar)`
firstvar

## [1] 20

# we can put basically anything into a variable, and we can call a variable
# pretty much whatever we want (but do avoid special characters besides "_")
myvar <- "boosh!"
myvar

myVar <- 5.5
myVar

## [1] "boosh!"
## [1] 5.5

# including other variables or computations involving them:
my_var <- myvar
my_var

myvar0 <- myVar / (myVar * 1.5)
myvar0

## [1] "boosh!"
## [1] 0.6666667

# when you introduce variables, they'll appear in the environment tab of the 
# top-right pane in R Studio. you can remove variables you're no longer
# using with `rm()`. (this isn't necessary, but it saves space in both 
# your brain and your computer's
rm(myvar)
rm(my_var)
rm(myVar)
rm(myvar0)

# R was designed with statistical applications in mind, so naturally there's
# lots of ways to represent collections or sequences of values (e.g. numbers).

# in R, a "vector" is the simplest list-like data structure.
# you can create a vector with the `c()` function (for "concatenate")
myvec <- c(1, 2, 3, 4, 5)
myvec

## [1] 1 2 3 4 5

anothervec <- c(4.5, 4.12, 1.0, 7.99)
anothervec

## [1] 4.50 4.12 1.00 7.99

# vectors can hold elements of any type, but they must all be of the same type.
# to keep things straight in your head, maybe include the data type in the name
myvec_char <- c("a","b","c","d","e")
myvec_char

## [1] "a" "b" "c" "d" "e"

# if we try the following, R will coerce the numbers into characters:
myvec2 <- c("a","b","c",1,2,3)
myvec2

## [1] "a" "b" "c" "1" "2" "3"

rm(myvec2)

suppose the only reason we created myvec and anothervec was to put them together with some other stuff, and save that to longvec. in this case, we can just remove myvec and anothervec, and use longvec

# you can put vectors or values together with `c()`
longvec <- c(0, myvec, 9, 80, anothervec, 0, NA)

rm(myvec)
rm(anothervec)

longvec

##  [1]  0.00  1.00  2.00  3.00  4.00  5.00  9.00 80.00  4.50  4.12  1.00
## [12]  7.99  0.00    NA

# to see how many elements a vector has, get its `length()`
length(longvec)

## [1] 14

# to see what the unique values are, use `unique()` (you'll get a vector back)
unique(longvec)

##  [1]  0.00  1.00  2.00  3.00  4.00  5.00  9.00 80.00  4.50  4.12  7.99
## [12]    NA

# a very common operation is to see how many unique values there are:
blah <- length(unique(longvec))

# to see a frequency table over a vector, use `table()`
table(longvec)

## longvec
##    0    1    2    3    4 4.12  4.5    5 7.99    9   80 
##    2    2    1    1    1    1    1    1    1    1    1

# note that this works for all kinds of vectors
table(c("a","b","c","b","b","b","a"))

## 
## a b c 
## 2 4 1

table(c(TRUE, FALSE, FALSE, FALSE, TRUE, FALSE))

## 
## FALSE  TRUE 
##     4     2

an important but not obvious thing:

R has a special value called NA, which represents missing data.

by default, table() won't tell you about NA's (annoying, ik!). so get in the habit of specifying the useNA argument of table()

# note "DRY" principle (don't repeat yourself)
vec_with_NA <- c(1,2,3,2,2,NA,3,NA,NA,1,1)
table(vec_with_NA)

## vec_with_NA
## 1 2 3 
## 3 3 2

table(vec_with_NA, useNA="ifany") # or "always" or "no"

## vec_with_NA
##    1    2    3 <NA> 
##    3    3    2    3

notice that the structure of the last table command is:

table(VECTOR, useNA=CHARACTERSTRING)

some terminology:

• table() is a function

• table() has argument positions for a vector and for a string

• we provided table() with two arguments:

  • a vector (vec_with_NA)
  • a character string ("ifany")

• the second argument position was named useNA

• we used the argument binding syntax useNA="ifany"

argument-binding is kind of like variable assignment, but useNA doesn't become directly available for use after we give it a value.

this might feel kinda abstract, but i promise the intuition will become clearer the further along we get.

some arguments – like useNA here – can be thought of as "options" of the function they belong to.

# here's an example that might clarify the concept of argument binding:
round(3.141592653, digits=4)

## [1] 3.1416

round() is a commonly used function that illustrates an important concept called vectorization.

many functions in R are vectorized by default, which means that they can take an individual value (like the round() call above), or they can take a vector of values.

in the latter case, the function applies pointwise to each element of the vector, and returns a vector with the same length and order as the input:

round(longvec, digits=4)

##  [1]  0.00  1.00  2.00  3.00  4.00  5.00  9.00 80.00  4.50  4.12  1.00
## [12]  7.99  0.00    NA

# if we don't tell it how many digits to round to, it defaults to 0
round(longvec)

##  [1]  0  1  2  3  4  5  9 80  4  4  1  8  0 NA

in fact MOST MATH STUFF IS VECTORIZED

we very often will want to access individual elements or subsets of a vector (e.g. if we've sorted a vector and want to look at its first element)

there are several ways to do this. here are some examples to give you an idea (note that 1:5 is the vector c(1,2,3,4,5), and == is "equals")

# a vector of several words
vec_words <- c("first","second","third","fourth","fifth")
vec_words[1]
vec_words[2:3]
vec_words[c(1,4)]
vec_words[vec_words=="first"]

## [1] "first"
## [1] "second" "third" 
## [1] "first"  "fourth"
## [1] "first"

let's inspect these a bit…

it's annoying to have to type every element of a vector. fortunately, there are many functions designed to make this unnecessary. for example rep() is short for "repeat"; seq() is short for "sequence"; letters is a built-in constant for the vector c("a","b","c",...)); and we just saw the range operator :.

# you can also combine times and each inside of rep()
# putting parentheses around an assignment causes it to print
(vec_num <- rep(1:5, times=2))

##  [1] 1 2 3 4 5 1 2 3 4 5

(vec_abc <- rep(letters[1:5], each=2))

##  [1] "a" "a" "b" "b" "c" "c" "d" "d" "e" "e"

(vec_odd <- seq(from=1, to=19, by=2))

##  [1]  1  3  5  7  9 11 13 15 17 19

very often we'll want to e.g. get the average value or the sum of a vector. we'll get way more into this in future sessions, but here's a preview:

# get the mean with mean()
(vec_num_mean <- mean(vec_num))
# or calculate it ourselves!
(vec_num_mean <- sum(vec_num) / length(vec_num))

# get the variance with var()
(vec_num_var <- var(vec_num))
# or calculate it ourselves
(vec_num_var <- sum((vec_num - mean(vec_num))^2)/(length(vec_num) - 1))

# get the correlation between vec_num and vec_odd
cor(vec_num, vec_odd, method="pearson")

## [1] 3
## [1] 3
## [1] 2.222222
## [1] 2.222222
## [1] 0.492366

exercise: compute the correlation between vec_num and vec_odd using only arithmetic.

here's an analogy to keep in mind: vectors are like columns of an abstract spreadsheet (not like rows).

• all their elements have to have the same type
• they have a length and you can perform operations on them
• they can contain missing values (NA)

in fact, this is a bit more than an analogy in R!

R's implementation of a "spreadsheet" – the data frame – is quite literally a list of vectors. the data frame is a beautiful data structure, and is used to represent (flat) datasets e.g. the contents of an excel sheet.

we'll have a first look at data frames next

a data frame is a list of vectors all of which have the same length.

first_df <- data.frame(1:5, letters[1:5], c(TRUE, TRUE, FALSE, NA, FALSE))

here's a slightly more interesting data frame, with names for columns.

(cool_df <- data.frame(
  id=paste0("id_", 1:6),             # identifier
  group=rep(c("a","b"), each=3),     # "a" = NYU law school, "b" = Columbia
  score=runif(n=6, min=1, max=100)   # score on the NY bar exam
))

##     id group    score
## 1 id_1     a 60.58062
## 2 id_2     a 72.88359
## 3 id_3     a 88.75228
## 4 id_4     b 68.73532
## 5 id_5     b 31.84391
## 6 id_6     b 46.13059

we can access rows or columns using square-bracket syntax [ , ]. the $ for individual columns is nice too – that gives us back a vector. lots of ways to slice + dice a df – below are some examples.

cool_df[1:3, ]

##     id group    score
## 1 id_1     a 60.58062
## 2 id_2     a 72.88359
## 3 id_3     a 88.75228

cool_df[, 1]

## [1] id_1 id_2 id_3 id_4 id_5 id_6
## Levels: id_1 id_2 id_3 id_4 id_5 id_6

cool_df$score

## [1] 60.58062 72.88359 88.75228 68.73532 31.84391 46.13059

cool_df[["score"]]

## [1] 60.58062 72.88359 88.75228 68.73532 31.84391 46.13059

cool_df[, "score"]

## [1] 60.58062 72.88359 88.75228 68.73532 31.84391 46.13059

cool_df[, c("id","score")]

##     id    score
## 1 id_1 60.58062
## 2 id_2 72.88359
## 3 id_3 88.75228
## 4 id_4 68.73532
## 5 id_5 31.84391
## 6 id_6 46.13059

# see who passed the exam:
cool_df$id[cool_df$score >= 60]

## [1] id_1 id_2 id_3 id_4
## Levels: id_1 id_2 id_3 id_4 id_5 id_6

we can add columns to a data frame by combining assignment <- with the dollar-sign $ column-grabbing syntax:

cool_df$passed <- ifelse(cool_df$score > 60, TRUE, FALSE)
cool_df$aced   <- ifelse(cool_df$score >= 90, TRUE, FALSE)
cool_df$failed <- !cool_df$passed

• exercise: what percent of the law students aced the exam?

• exercise: what was the average score for each group?

finally, some useful functions to use on data frames to check them out a bit. (note that you can combine names() with assignment to change the column names)

head(cool_df, n=2)
dim(cool_df)        # a vector of length 2: number of rows, number of cols
nrow(cool_df)       # number of rows
ncol(cool_df)       # number of columns
names(cool_df)      # the names of the columns

##     id group    score passed  aced failed
## 1 id_1     a 60.58062   TRUE FALSE  FALSE
## 2 id_2     a 72.88359   TRUE FALSE  FALSE
## [1] 6 6
## [1] 6
## [1] 6
## [1] "id"     "group"  "score"  "passed" "aced"   "failed"

str(cool_df)        # the structure of the data frame
summary(cool_df)    # gives useful info about each column

## 'data.frame':    6 obs. of  6 variables:
##  $ id    : Factor w/ 6 levels "id_1","id_2",..: 1 2 3 4 5 6
##  $ group : Factor w/ 2 levels "a","b": 1 1 1 2 2 2
##  $ score : num  60.6 72.9 88.8 68.7 31.8 ...
##  $ passed: logi  TRUE TRUE TRUE TRUE FALSE FALSE
##  $ aced  : logi  FALSE FALSE FALSE FALSE FALSE FALSE
##  $ failed: logi  FALSE FALSE FALSE FALSE TRUE TRUE
##     id    group     score         passed           aced        
##  id_1:1   a:3   Min.   :31.84   Mode :logical   Mode :logical  
##  id_2:1   b:3   1st Qu.:49.74   FALSE:2         FALSE:6        
##  id_3:1         Median :64.66   TRUE :4                        
##  id_4:1         Mean   :61.49                                  
##  id_5:1         3rd Qu.:71.85                                  
##  id_6:1         Max.   :88.75                                  
##    failed       
##  Mode :logical  
##  FALSE:4        
##  TRUE :2        
##                 
##                 
## 

• installing and loading packages
• more on data frames (factors, character, etc.)
• reading in external datasets as data frames
• manipulating data frames
• cleaning up data frames
• summarizing them